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INVESTIGATING 
SCHEDULE  SLIPPAGE 


CAPT  JAMES  V.  MONACO »  USAF  AND 
EDWARD  D.  WHITE  III ,  PH.D 


Past  research  shows  that  schedule  slippage  within  the  acquisition 
community  often  adversely  affects  the  cost  and  performance  charac¬ 
teristics  within  a  program.  To  minimize  the  risk  of  underestimating 
schedule  growth,  a  program  manager  needs  a  reliable  initial  schedule 
estimate.  Statistical  models  can  provide  such  estimates;  however,  they 
require  accurate  historical  data  and  predictive  drivers.  Many  archival 
studies  have  investigated  potential  drivers  of  schedule  growth.  In  this 
article,  we  review  several  of  those  studies  that  investigated  schedule 
slippage  and  highlight  common  potential  drivers  of  schedule  growth, 
ending  with  a  list  of  variables  for  estimators  to  consider  for  incorporating 
into  future  predictive  models. 


One  could  compare  a  System-of-Systems  (SOS)  approach  to  building  a  pyra¬ 
mid.  As  with  any  pyramid,  the  top  is  only  as  strong  as  the  foundation.  With 
an  SOS  approach,  that  foundation  often  rests  with  the  individual  systems. 
This  in  turn  implies  that  any  risk  that  an  individual  system  may  be  subject  to  also 
makes  the  overall  system  vulnerable  as  well.  In  the  acquisition  world,  two  of  the 
biggest  risks  of  any  system  are  cost  growth  and  schedule  slippage,  which  often  results 
in  cost  growth. 

Developing  a  major  weapon  system  is  risky  and  full  of  uncertainty.  Requirements, 
politics,  economics,  and  the  system’s  technological  design  are  just  a  few  of  the 
uncertainties  that  create  risk  in  this  venture.  This  uncertainty  manifests  itself  in  the 
form  of  variance  between  the  planned  schedule  and  the  actual  schedule.  “Excessive 
schedules  have  two  significant  negative  effects:  U.S.  forces  may  be  left  without  needed 
capabilities,  and  longer  schedules  often  mean  higher  costs”  (Tyson,  Harmon,  &  Utech, 
1994,  p.  S-l).  The  accuracy  of  the  program  schedule  estimate,  therefore,  is  critical 
in  planning  for  future  capability  needs  and  correctly  accounting  for  the  cost  in 
developing  the  weapon  system. 
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To  gain  a  handhold  on  minimizing  the  risk  of  underestimating  a  program’s  actual 
acquisition  schedule,  one  first  needs  an  accurate  estimate  of  a  program’s  schedule. 
Statistical  regression  models  can  generate  such  estimates,  but  these  predictive  tools 
need  reliable  data  and  historical  trends  in  order  to  be  viable.  For  this  requisite  infor¬ 
mation,  researchers  often  pull  past  studies;  however,  this  can  be  cumbersome  if  not 
easily  found  or  at  least  grouped  together.  In  this  vein,  we  provide  an  overview  of 
some  past  research  conducted  on  schedule  variance.  We  describe  the  reasons  given 
for  schedule  slippage  within  the  framework  of  the  acquisition  environment  followed 
by  both  descriptive  and  inferential  statistical  studies  relating  to  schedule  overruns. 

From  these  studies,  we  go  on  to  tabulate  some  of  the  key  drivers  associated  with 
schedule  slippage.  These  enable  researchers  to  perhaps  gain  a  better  handhold  on 
what  could  cause  schedule  slippage.  This  in  turn  could  assist  them  in  building  a 
database  foundation  from  which  to  develop  models  to  predict  more  accurate  acquisition 
schedule  estimates  for  major  weapon  systems.  This  information  could  also  serve  as 
a  possible  flag  or  indicator  of  when  a  SOS  may  experience  schedule  slippage  itself. 


FACTORS  AFFECTING  SCHEDULE 

Factors  Affecting  Original  Plan 

1. 

Competition 

2. 

Concurrency  (overlap  of  effort  betwee 

n  development  and  production  phase) 

3. 

Funding  adequacy 

4. 

Inclusion  of  prototype  phase 

5. 

If  the  program's  phases  were  contracted  separately 

6. 

Service  priority 

(Drezner  &  Smith,  1 990,  pp.  21-22) 

Factors  Affecting  Program  Deviation 

1. 

Contractor  performance 

2. 

External  events 

3. 

Funding  stability 

4. 

Major  requirements  stability 

5. 

Program  manager  turnover 

(Drezner  &  Smith,  1 990,  pp.  23-24) 

Factors  Affecting  Original  Plan  and/or  Program  Deviation 

1. 

External  guidance 

2. 

Single  service  or  joint  management 

3. 

Program  complexity 

4. 

Technical  difficulty 

5. 

Concept  stability  (system  specification 

maturity) 

(Drezner  &  Smith,  1 990,  p.  23) 

FIGURE  1.  DREZNER  AND  SMITH'S  16  SCHEDULE  FACTORS 


178 


POTENTIAL  REASONS 


The  time  required  to  create  a  new  weapon  system  from  idea  inception  to  a  fielded 
system  is  an  important  element  to  understand  in  the  overall  acquisition  process.  Cost 
and  schedule  overruns  in  the  development  of  major  weapon  systems  are  continuing 
problems  that  plague  the  acquisition  environment.  The  following  studies  address  and 
discuss  some  of  the  reasons  for  these  overruns. 

DREZNER  AND  SMITH  STUDY,  1990 

Although  knowing  the  duration  and  variability  of  schedule  is  important,  under¬ 
standing  what  factors  make  up  the  duration  and  affect  the  variability  are  imperative. 
Based  on  statistical  analysis  of  10  programs,  the  results  of  a  1990  RAND  study  by 
Drezner  and  Smith  suggest  the  following  influences  on  the  original  schedule  estimate: 
1)  competition  and  prototyping  lengthens  schedule,  and  2)  concurrency  and  adequate 
funding  shortens  schedule.  Results  also  suggest  the  following  influences  on  schedule 
slips:  1)  unstable  funding,  2)  technical  difficulty,  3)  external  guidance,  and  4)  external 
events  (Drezner  &  Smith,  1990,  p.  33).  Two  commonly  held  hypotheses  that  prove 
inconclusive  for  Drezner  and  Smith,  though  the  small  sample  size  of  the  study  may 
be  a  limiting  factor,  are  that  longer  planning  phases  incur  less  slippage  and  that  cost 
and  schedule  growth  are  interrelated.  Figure  1  lists  the  major  attributing  factors 
addressed  by  this  study. 

TYSON,  NELSON,  OM,  AND  PALMER  STUDY,  1989 

The  study,  conducted  by  the  Institute  for  Defense  Analyses  (IDA),  examined  schedule 
variances  and  its  causes.  The  study’s  database  consisted  of  9  tactical  aircraft,  9 
electronic  aircraft,  5  helicopters,  8  other  aircraft,  16  air-launched  tactical  munitions, 
18  surface-launched  tactical  munitions,  10  electronic  systems,  10  strategic  missiles, 
and  4  satellites.  The  main  focus  of  this  study  was  to  determine  the  effect,  if  any,  on 
schedule  overruns  from:  1)  prototyping,  2)  competition,  3)  multiyear  procurement,  4) 
design-to-cost,  5)  sole-source  procurement  and  fixed-price  development,  and  6)  con¬ 
tract  incentives.  This  is  what  the  researchers  found. 

Use  of  prototyping  showed  a  reduction  in  the  development  phase  and  the  overall 
schedule  by  1 1  and  15  percent  respectively  (Tyson,  Nelson,  Om,  &  Palmer,  1989,  pp. 
VIII-6-VIII-7).  Competitive  programs  produced  43  percent  more  design-schedule 
growth  and  39  percent  more  production-schedule  growth  compared  to  noncompeti¬ 
tive  programs  (Tyson  et  al.,  1989,  p.  VII-7).  Programs  using  multiyear  procurement 
experienced  7  percent  less  production  schedule  growth  (Tyson  et  al.,  1989,  p.  VI-8). 
Design- to-cost  exhibited  development  schedule  growth  of  12  percent  and  production 
schedule  growth  of  2  percent  (Tyson  et  al.,  1989,  p.  IX-11).  Production  schedule 
growth  was  reduced  by  27  percent  when  sole-source  procurement  is  used  (Tyson  et 
al.,  1989,  p.  X-7).  Under  a  fixed-price  contract  strategy,  development  schedule  growth 
showed  a  reduction  of  6  percent  (Tyson  et  al.,  1989,  p.  X-13).  Even  though  contract 
incentives  were  one  of  the  focuses  for  this  study,  no  comparison  was  made  between 
contract  incentives  and  schedule. 
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JOINT  STRIKE  FIGHTER  (JSF)  STUDY,  2000 

The  Department  of  Defense  (DoD)  selected  the  Joint  Strike  Fighter  (JSF)  program 
as  a  flagship  program  for  acquisition  reform  with  a  principle  objective  to  demonstrate 
a  low  level  of  technical  risk  for  critical  technologies  in  the  concept  demonstration 
phase  (Rodrigues,  2000,  p.  1).  This  strategy  recommends  focusing  on  risk  reduction 
efforts  by  maturing  critical  technologies  prior  to  entering  Engineering  and  Manufac¬ 
turing  Development  (EMD),  and  being  allowed  to  do  so  without  the  penalty  of  re¬ 
duction  in  funding  support  (Rodrigues,  2000,  p.  3). 

This  study  addressed  the  JSF’s  acquisition  strategy  and  noted  it  as  encouraging, 
but  also  stated  that  program  managers  are  hesitant  to  embrace  this  concept.  Once  in 
the  development  environment,  external  pressure  to  keep  the  program  moving  becomes 
dominant,  such  as  preserving  cost  and  schedule  estimates  to  secure  budget  approval. 
If  a  program  manager  decides  that  an  additional  year  is  needed  in  the  risk  reduction 
or  concept  development  stage  to  reach  the  desired  level  of  technical  maturity,  they 
run  the  risk  of  reduced  funding  for  the  development  stage,  which  could  lead  to  program 
cancellation  (Rodrigues,  2000,  p.  6).  They  are  more  likely  to  accept  a  lower  level  of 
technology  rather  than  risking  the  program.  Unfortunately,  low  levels  of  maturity  lead 
to  increased  risks,  which  in  turn  lead  to  the  likelihood  of  schedule  delays,  increased 
costs,  and/or  quantity  reduction. 


DESCRIPTIVE  STUDIES 

The  previous  section  highlighted  a  few  studies  that  identified  reasons  for  schedule 
slippage.  They  essentially  answered  the  question  of  what  causes  schedules  to  slip. 
Next,  we  proceed  to  relay  both  descriptive  and  inferential  statistical  studies  that  answer 
the  questions  of  how  and  why  schedule  growth  occurs  within  the  acquisition  community. 

REIG  STUDY,  1995 

In  a  study  to  identify  the  track  record  of  success  for  acquisition  programs  based 
on  the  number  of  Low  Rate  Initial  Production  (LRIP)  articles  used  for  testing,  the 
office  of  the  Director  of  Operational  Test  and  Evaluation  asked  the  Defense  Systems 
Management  College  to  research  the  current  acquisition  system  (Reig,  1995,  p.  27). 
The  question  that  the  researchers  wanted  to  answer  was  the  following:  Is  there  a 
relationship  between  the  number  of  LRIP  articles  used  for  testing  in  EMD  and  the 
success  of  that  program? 

The  researchers  analyzed  24  programs  of  all  types,  concentrating  exclusively  on 
the  EMD  phase.  None  of  the  24  programs  studied  completed  the  EMD  phase  of 
acquisition  within  the  planned  schedule,  and  the  average  schedule  slippage  was  63 
percent  (Reig,  1995,  p.  28).  For  data  they  used  the  Selected  Acquisition  Report  (SAR) 
database  and  a  review  of  the  Blue  Books,  which  are  summary  program  plans  and  data 
compiled  for  the  use  of  Defense  Acquisition  Board  principles  and  staff  assistants 
immediately  prior  to  milestone  meetings  (Reig,  1995,  p.  27). 
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The  results  of  the  study  showed  that  the  percentage  of  LRJP  test  quantity  (LRIP 
test  quantity  divided  by  total  planned  production  quantity)  is  an  effective  predictor 
of  the  success  of  that  program.  That  is,  programs  using  three  percent  or  more  LRIP 
test  articles  never  exceeded  a  50  percent  slip  and  the  percentage  of  slip  steadily 
decreased  as  the  three-percent  mark  was  approached  (Reig,  1995,  p.  28).  The  LRIP 
was  not  the  only  variable  to  potentially  explain  the  success  of  a  program.  The  study 
identified  six  other  explanatory  variables.  These  included  the  degree  of  risk  identi¬ 
fied  at  Milestone  II,  use  of  competition  at  Demonstration/Validation  (DEM/VAL) 
phase,  use  of  competition  during  EMD  phase,  contract  type,  number  of  associate 
contractors,  and  joint-  or  single-Service  program  (Reig,  1995,  p.  29). 


The  results  of  the  study  showed  that  the 
pertentage  of  LRIP  test  quantity  (LRIP  test 
quantity  divided  by  total  planned  produ<tion 
quantity)  is  an  effettive  predittor  of 
the  su<tess  of  that  program. 


For  these  variables,  this  is  what  the  researchers  concluded.  Programs  that  were 
medium-risk  at  Milestone  II  had  a  higher  average  program  success  than  those  that 
were  low-risk.  Programs  that  did  not  use  competition  in  DEM/VAL  or  in  EMD  had 
a  higher  success  rate.  As  far  as  contract  type,  Cost  Plus  Incentive  Fee  (CPIF),  Fixed 
Price  Incentive  (FPI),  and  Firm  Fixed  Price  (FFP)  had  equally  higher  program  success 
in  comparison  to  Cost  Plus  Award  Fee  (CPAF).  The  data  also  showed  that  programs 
using  one  or  fewer  associate  contractors  (besides  the  Prime)  had  a  higher  pro¬ 
gram  success  than  those  that  used  more  than  one  associate  contractor.  Similarly, 
single-Service  programs  had  higher  program  success  compared  to  joint  programs 
(Reig,  1995,  p.  29). 

JSF  STUDY,  2000 

In  the  JSF  study  previously  mentioned,  it  also  discussed  the  use  of  technology 
readiness  levels  (TRLs).  Pioneered  by  the  National  Aeronautics  and  Space  Admin¬ 
istration  (NASA)  and  adopted  by  the  Air  Force  Research  Laboratory  (AFRL),  TLRs 
are  used  to  determine  the  readiness  of  technologies  incorporated  into  a  weapon  or 
other  type  of  system  (Rodrigues,  2000,  p.  9).  When  measured  on  a  scale  of  one  to 
nine,  the  lower  the  level  of  maturity  when  a  technology  is  included  in  a  development 
program  the  higher  the  risk  that  it  will  cause  problems  such  as  schedule  delays  in  the 
future  (Rodrigues,  2000,  p.  8).  According  to  NASA,  AFRL,  and  others  in  DoD,  a 
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level  of  seven  enables  a  technology  to  be  included  in  a  development  program  with 
acceptable  risk  (Rodrigues,  2000,  p.  9). 

Prior  to  the  JSF,  TRLs  were  also  used  to  assess  the  impact  of  technological  maturity 
on  product  outcomes.  The  JSF  study  reviewed  23  different  technologies  incorporated 
into  new  product  and  weapon  systems  designs,  either  within  DoD  or  the  commercial 
sector,  and  found  the  following.  First,  that  cost  and  schedule  problems  arise  when 
programs  start  with  technologies  at  low  readiness  levels.  And  second,  it  conversely 
showed  that  programs  met  product  objectives  when  the  technologies  are  at  higher 
levels  of  readiness. 

GAILEY  STUDY,  2002 

As  explained  earlier,  the  1995  Reig  study  examined  preliminary  data  using  infor¬ 
mation  from  24  acquisition  programs  to  determine  if  any  relationship  between  LRIP 
quantities  and  the  success  of  the  program  in  EMD  existed.  This  2002  study  expanded 
the  dataset  to  include  46  programs  that  had  completed  EMD  (Gailey,  2002,  p.  5). 
Contrary  to  the  results  of  Reig  (1995),  which  used  a  smaller  database,  this  study 
detected  no  correlation  between  LRIP  quantities  and  the  probability  that  the  schedule 
will  slip  (Gailey,  2002,  p.  5).  Additionally,  no  differences  were  noted  in  EMD  success 
attributable  to  whether  EMD  is  competed,  how  many  associated  contractors  are  present, 
or  whether  the  program  is  joint-Service  (Gailey,  2002,  p.  9).  Lastly,  the  findings  of 
this  study  also  reiterated  that  programs  that  did  not  compete  the  DEM/VAL  phase 
had  higher  program  success  and  narrowed  the  best  contract  type  to  CPIF.  That  is, 
CPIF  contracts  in  EMD  produced  greater  success  than  CPAF,  FPI,  or  FFP  contracts 
(Gailey,  2002,  p.  9). 


INFERENTIAL  STUDIES 

THE  ANALYTIC  SCIENCES  CORPORATION  (TASC)  STUDIES,  1986  &  1987 

In  the  1980s,  the  Air  Force’s  Aeronautical  System  Division  (ASD)  called  upon  The 
Analytic  Sciences  Corporation  (TASC)  to  improve  ASD’s  independent  schedule 
assessment  capabilities  for  the  Full  Scale  Development  (now  known  as  EMD) 
phase.  Although  various  techniques  using  models  based  upon  Cost  Estimating 
Relationships  (CERs)  provide  a  cost  estimate  from  data  of  analogous  systems,  the 
same  level  of  attention  was  not  given  to  Scheduling  Estimating  Relationships  (SERs) 
(Nelson,  1986,  p.  1-1). 

TASC  developed  a  schedule  database  consisting  of  17  aircraft,  conducted  a  litera¬ 
ture  review,  and  identified  potential  drivers.  They  noted  that  the  popular  drivers  used 
in  CERs,  such  as  continuous  physical  parameters,  rather  than  discrete  categorical 
variables,  showed  little  significance  in  the  use  of  SERs  (Nelson,  1986,  p.  3-2).  The 
TASC  broke  the  EMD  phase  into  six  duration  intervals.  These  intervals  were:  1) 
Milestone  II  to  EMD  contract  award,  2)  EMD  contract  award  to  first  flight,  3)  first 
flight  to  first  production  unit,  4)  first  production  unit  to  Initial  Operational  Capability 
(IOC),  5)  EMD  contract  award  to  IOC,  and  (6)  Milestone  II  to  IOC  (Nelson  &  Trageser, 
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1987,  pp.  2-6).  The  TASC  also  grouped  potential  schedule  drivers  into  six  distinct 
categories:  1)  technical  complexity,  2)  degree  of  technological  change,  3)  system 
mission,  4)  period  of  procurement,  5)  acquisition  strategy,  and  6)  funding  profile 
(Nelson  &  Trageser,  1987,  pp.  2-8).  The  TASC  then  conducted  a  correlation  analysis 
between  these  durations  and  category  drivers. 


The  purpose  of  the  study  was  to  provide  a  point 
estimate  and  range  for  the  expetted  sthedule 
duration  of  future  programs  by  treating 
probability  distributions  of  past  sthedule 
durations  within  tertain  intervals. 


For  the  two  schedule  durations,  Milestone  II  to  IOC  and  EMD  contract  award  to 
IOC,  the  researchers  suggested  using  mission  type  (cargo,  tanker,  attack,  or  fighter), 
avionics  complexity  (low,  medium,  or  high),  and  the  period  of  acquisition  for  the 
analysis  (Nelson  &  Trageser,  1987,  pp.  4-5).  Their  findings:  A  positive  correlation 
exists  between  mission  type  and  schedule  duration  where  the  schedule  durations 
increase  from  cargo  to  fighter  aircraft  (Nelson  &  Trageser,  1987,  pp.  2-17).  Addition¬ 
ally,  the  period  of  acquisition  also  showed  a  positive  correlation  between  the  year 
development  started  and  the  schedule  duration. 

Much  like  the  findings  in  the  1990  RAND  study,  schedule  duration  within  EMD 
overall  showed  an  increase  from  past  to  present  (Nelson  &  Trageser,  1987,  pp.  2-18; 
Drezner  &  Smith,  1990,  p.  11).  Another  finding  of  this  study,  which  contrasts  that 
of  later  studies  by  RAND  and  IDA,  stated  that  no  significant  relationships  exist  between 
prototyping  and  EMD  schedule  durations  (Nelson  &  Trageser,  1987,  pp.  3-13;  Tyson 
et  al.,  1989,  pp.  VIII6-VIII-7;  Harmon,  Ward,  &  Palmer,  1989,  p.  47;  and  Drezner 
&  Smith,  1990,  p.  30). 

FITCHER,  ARNOLD,  AND  ALLEN  STUDY,  1992 

This  study  presented  a  historical  perspective  of  DoD  program  schedule  perfor¬ 
mance,  based  on  35  Army,  46  Navy,  and  24  Air  Force  programs  from  the  December 
1991  Selected  Acquisition  Reports  (Fitcher,  Arnold,  &  Allen,  1992,  p.  1).  The  purpose 
of  the  study  was  to  provide  a  point  estimate  and  range  for  the  expected  schedule 
duration  of  future  programs  by  creating  probability  distributions  of  past  schedule 
durations  within  certain  intervals.  The  intervals  were:  1)  Milestone  I  -  Milestone  II, 
2)  Milestone  II  -  Milestone  III,  3)  program  start  to  first  flight,  4)  program  start  to 
first  unit  equipped,  and  5)  program  start  to  IOC  (Fitcher,  Arnold,  &  Allen,  1992,  p.  1). 
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Although  this  study  in  no  way  tried  to  predict  the  schedule  duration  of  a  specific 
interval  based  on  predictor  variables,  it  did  provide  an  ability  to  check  the  realism 
of  schedules  proposed  by  the  program  managers.  The  probability  distributions  were 
compared  by  service  and  by  intervals,  to  give  a  range  of  values  as  “most  likely”  and 
schedule  expectations  considered  overly  optimistic  or  pessimistic  (Fitcher  et  al.,  1992, 
p.  2).  Results  from  this  study  showed  that  no  marked  difference  exists  among  the  data 
from  within  each  Service.  Flowever,  the  study  noted  that  the  only  significant  differ¬ 
ence  among  the  Services  (at  an  alpha  level  of  0.05)  is  a  longer  than  average  time  for 
Air  Force  programs  compared  to  the  Army  and  Navy  between  Milestone  II  and 
Milestone  TIT. 

HARMON,  WARD,  AND  PALMER,  1989 

The  IDA  attempted  to  provide  methods  for  assessing  the  reasonableness  of  pro¬ 
posed  acquisition  schedules  in  the  first  of  four  studies.  This  particular  study,  consist¬ 
ing  of  data  collected  from  nine  tactical  aircraft  programs,  analyzed  schedule  intervals 
and  provided  a  schedule  assessment  tool  that  spanned  the  period  from  Full  Scale 
Development  (FSD)  (now  referred  to  as  EMD)  start  through  full-rate  production 
(Flarmon,  Ward,  &  Palmer,  1989,  p.  1).  The  programs  chosen  with  development 
occurring  from  the  early  1970s  to  early  1980s  were  based  on  the  newness  of  the 
program,  its  importance  in  historical  perspective,  and  the  expected  availability  of  data 
(Flarmon,  Ward,  &  Palmer,  1989,  p.  17).  The  researchers  obtained  cost  and  technical 
data  from  government  sources  and  prime  contractors,  while  schedule  data  were  acquired 
from  SARs,  contractors,  and  the  Services’  sources  (Flarmon,  Ward,  &  Palmer,  1989, 
pp.  17-18). 


Although  this  study  in  no  way  tried  to  predict 
the  schedule  duration  of  a  specific  interval  based 
on  predictor  variables,  it  did  provide  an  ability 
to  check  the  realism  of  scheduies  proposed 
by  the  program  managers . 


The  researchers  also  gathered  data  on  certain  program  and  aircraft  characteristics, 
such  as  contractor,  mission,  and  prototyping.  The  study  revealed  that  the  program 
attributes  play  an  important  role  in  explaining  variations  in  interval  length.  For  in¬ 
stance,  under  the  program  attribute  of  the  prime  contractor,  it  is  estimated  that 
McDonnell  Aircraft  programs  require  15  percent  more  time  than  other  contractor 
types  considered  (Flarmon  Ward,  &  Palmer,  1989,  p.  47).  Additionally,  the  data  also 
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showed  that  prototype  programs  require  1 1  percent  less  time  than  programs  that  do 
not  develop  prototypes  (Harmon  Ward,  &  Palmer,  1989,  p.  47). 

HARMON  AND  WARD,  1 989 

The  second  IDA  study  was  a  follow-up  to  the  previous  IDA  study  on  tactical  aircraft 
acquisition  schedules.  The  approach  used  in  this  study  in  many  ways  paralleled  that 
used  for  the  previous  study.  The  data  consisted  of  14  air-launched  missile  programs 
(seven  air-to-air  and  seven  air-to-surface  systems)  that  involved  substantial  develop¬ 
ments  from  the  mid  1960s  to  the  late  1980s  (Harmon  &  Ward,  1989,  p.  7).  The  data 
were  collected  from  military  services,  prime  contractors,  and  third  parties  (studies 
and  databases  at  IDA,  RAND,  and  others),  with  schedule  and  missiles  characteristic 
data  obtained  from  SARs,  numerous  government  sources,  and  secondary  data  sources 
(Harmon  &  Ward,  1989,  p.  8).  The  researchers  also  collected  data  on  program  and 
missile  characteristics. 


From  their  analysis,  the  resear<hers  believed  that 
the  most  important  determinant  of  overall 
development  program  length  is  length  of 
the  flight  test  program . 


The  researchers  originally  wanted  to  develop  a  single  equation  to  predict  the  interval 
of  FSD,  which  was  defined  as  the  period  from  FSD  start  to  delivery  of  the  first 
production  missile  (Harmon  &  Ward,  1989,  p.  23).  However,  due  to  the  fact  that  the 
determinants  of  time  to  first  launch  and  time  from  first  launch  to  first  production  are 
vastly  different,  the  researchers  chose  the  interval  between  first  guided-launch  and 
the  first  production  delivery  to  model  (Harmon  &  Ward,  1989,  p.  36).  From  their 
analysis,  the  researchers  believed  that  the  most  important  determinant  of  overall 
development  program  length  is  length  of  the  flight  test  program.  Because  flight  test 
duration  is  determined  by  the  number  of  test  missiles  launched  and  the  rate  at  which 
test  launches  are  accomplished,  it  is  no  surprise  that  the  most  important  program 
attribute  in  determining  development  effort  length  was  the  number  of  missiles  launched 
during  flight  tests  (Harmon  &  Ward,  1989,  p.  13). 

HARMON  AND  OM  STUDY,  1 993 

The  third  of  four  studies  by  IDA,  this  study’s  approach  was  consistent  with  the 
previous  two  studies  but  differed  in  what  was  studied.  The  data  collected  consisted 
of  the  most  important  26  unmanned  spacecraft  programs  spanning  the  timeframe  of 
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the  late  1960s  to  the  early  1990s.  Included  in  the  dataset  were  operational  DoD 
spacecraft,  four  commercial  communications  spacecraft,  and  selected  NASA  and  DoD 
experimental  and  scientific  unmanned  spacecraft  (Harmon  &  Om,  1993,  p.  1-2). 
Although  the  original  approach  to  this  analysis  of  development  schedule  data  was  to 
partition  the  schedule  into  multiple  intervals,  spacecraft  development  program  sched¬ 
ule  intervals  were  not  characterized  consistently  enough  to  yield  satisfactory  estimat¬ 
ing  relationships  (Harmon  &  Om,  1993,  p.  1-2).  Instead,  the  researchers  chose  to 
estimate  development  schedule  as  a  whole.  Development  schedule  duration  was  defined 
as  the  interval  between  the  authority  to  proceed  (ATP)  date  and  delivery  of  the  first 
flight-model  spacecraft  (Harmon  &  Om,  1993,  p.  1-3). 

In  order  to  compare  across  all  programs,  the  milestone  dates  were  standardized. 
For  DoD  spacecraft,  ATP  generally  equated  to  the  start  of  EMD.  For  NASA  spacecraft, 
it  meant  the  start  of  phase  design/development  C/D1  contract  (Harmon  &  Om,  1993, 
p.  III-7).  The  26  programs  used  in  this  study  were  grouped  into  one  of  three  categories: 
DoD  sensor/navigation  (8  spacecraft),  NASA  scientific/experimental  (9  spacecraft), 
or  operational  communications  (9  spacecraft)  (Harmon  &  Om,  1993,  p.  III-2).  In 
analyzing  the  schedule  interval  data,  the  researchers  tested  relationships  for  intervals 
at  more  disaggregated  levels  than  EMD  as  a  whole,  such  as  time  from  development 
start  to  critical  design  review  (CDR),  and  time  from  CDR  to  first  production  delivery 
(Harmon  &  Om,  1993,  p.  IV-2). 


This  IDA  study  was  the  last  of  four  sequenced 
studies  and  built  on  the  three  previous  IDA 
studies  of  tactical  aircraft ,  air-launched  missiles, 
and  unmanned  spacecraft  acquisition  schedules. 


Unfortunately,  the  regression  models  for  decomposed  timeframes  proved  less  than 
satisfactory,  due  to  missing  data,  inconsistency  of  data  between  the  three  program 
categories,  and  large  unexplained  variability  for  the  NASA  data  points  (Harmon  & 
Om,  1993,  p.  IV-23).  On  the  positive  side,  the  measures  of  spacecraft  size  and 
complexity  proved  to  be  good  explanatory  variables,  with  the  variable,  “Beginning  of 
life  power  in  watts,”  proving  most  satisfying  (Harmon  &  Om,  1993,  p.  IV-4).  The 
researchers  also  stated  that  all  the  categorical  variables  for  sensor,  navigation,  planetary, 
and  commercial  spacecraft  were  statistically  significant  at  an  alpha  level  of  0.03  or 
better  (Harmon  &  Om,  1993,  p.  IV-4). 

HARMON  AND  OM  STUDY,  1 995 

This  IDA  study  was  the  last  of  four  sequenced  studies  and  built  on  the  three  previous 
IDA  studies  of  tactical  aircraft,  air-launched  missiles,  and  unmanned  spacecraft 
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acquisition  schedules.  The  data  collected  consisted  of  22  missile  programs  with 
substantial  developments  from  the  mid  1960s  to  the  1990s.  These  programs  included 
eight  surface-launched  interceptors,  seven  air-launched  interceptors,  and  seven  air- 
launched  surface-attack  missiles  (Harmon,  &  Om,  1995,  p.  1-2).  Although  the  focus 
of  this  study  was  on  interceptor  missiles,  inclusion  of  the  attack  missiles  was  used 
because  attack  missile  programs  tend  to  be  influenced  by  the  same  drivers  and  the 
missiles’  hardware  also  share  many  attributes  (Harmon,  &  Om,  1995,  p.  II-l). 

In  the  previous  study  of  air-launched  missile  programs,  the  researchers  used  the 
delivery  date  of  the  first  production  missile  to  mark  the  end  of  development,  due  to 
wide  variance  in  production  rates  associated  with  different  types  of  missiles.  The  data 
for  this  study  encountered  the  same  variability  and  therefore  used  this  same  date  to 
mark  the  end  of  development  (Harmon  &  Om,  1995,  pp.  1-3— 1-4).  Although  emphasis 
was  placed  on  both  the  pre-EMD  and  the  EMD  phase  of  the  acquisition  cycle,  schedule 
intervals  in  the  concept  exploration  phases  and  the  demonstration  and  validation  phase 
prior  to  EMD  were  often  highly  dependent  upon  political  factors  and  therefore  not 
emphasized.  Pre-EMD  prototype  intervals,  on  the  other  hand,  were  an  exception 
(Harmon  &  Om,  1995,  p.  II-l). 


In  the  previous  study  of  air-lauiuhed  missile 
programs,  the  researchers  used  the  delivery  date 
of  the  first  produ<tion  missile  to  mark  the  end  of 
development,  due  to  wide  variance  in  produ<tion 
rates  assotiated  with  different  types  of  missiles . 


The  researchers  originally  wanted  to  develop  a  single  equation  to  estimate  the 
interval  of  EMD  defined  as  the  period  from  EMD  start  to  delivery  of  the  first 
production  missile  (Harmon  &  Om,  1995,  p.  Ill- 1).  “Unfortunately,  the  determinants 
of  time  to  first  launch  and  time  from  first  launch  to  first  production  are  just  too 
different”  (Harmon  &  Om,  1995,  p.  III-24).  Therefore,  the  researchers  instead  chose 
the  interval  between  first  guided-launch  and  the  first  production  delivery  as  the  interval 
to  model  (Harmon  &  Om,  1995,  p.  Ill- 1).  Their  findings  were  similar  to  the  1993 
study  by  IDA  on  spacecraft — the  most  important  program  attribute  in  determining 
length  of  the  development  effort  is  the  number  of  missiles  launched  during  flight 
tests  (Harmon  &  Om,  1995,  p.  II-9). 

TYSON,  HARMON,  AND  UTECH  STUDY,  1994 

Unrelated  to  the  four  previous  studies,  IDA  performed  an  analysis  on  20  tactical 
missile  and  7  tactical  aircraft  programs  with  the  objective  to  describe  costs  and  schedule 
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Occurrence  of  Predictive 

Schedule  Drivers  in  Literature 

Predictive  Schedule  Drivers 

Year 

1987 

1989 

1989 

1989 

1990 

1992 

1993 

1994 

1995 

1995 

2000 

2002 

Competition 

Concurrency 

Contract  type 

Design-to-cost 

External  events/guidance 
Funding  (amount/stability) 
Increase  in  quantity 

Length  of  testing 

LRIP/number  of  test  articles 
Mission  type 

Multiyear  procurement 

Number  of  contractors 

Period  of  acquisition 

Prime  contractor 

Program  size 

Prototyping 

Service  type  (joint  or  single) 
Sole  source  procurement 
Technical  issues 
(maturity/difficulty/complexity) 

X 

X 

X 

X 

4 

X 

X 

2 

X 

X 

X 

3 

X 

1 

X 

1 

X 

1 

X 

1 

X 

X 

2 

X 

1 

X 

X 

2 

X 

1 

X 

1 

X 

1 

X 

1 

X 

1 

X 

X 

X 

3 

X 

X 

2 

X 

1 

X 

X 

X 

X 

X 

5 

TABLE  1.  PREDICTIVE  SCHEDULE  DRIVERS  IN  LITERATURE 

growth  patterns  associated  with  the  acquisition  of  selected  major  systems.  Moreover, 
they  wanted  to  identify  reasons  for  the  growth  and  then  develop  a  way  to  predict 
growth  in  ongoing  development  and  early  production  phases  (Tyson,  Harmon,  &  Utech, 
1994,  p.  iii).  Data  used  for  this  study  were  obtained  from  SARs,  historical  memo¬ 
randa  to  support  DoD  program  reviews,  and  from  summaries  of  program  data  (Tyson, 
Harmon,  &  Utech,  1994,  p.  S-l). 

The  study  found  that  programs  took  from  50  to  137  months  from  Milestone  II  to 
IOC  with  only  2  of  the  20  tactical  missile  programs  finishing  on  time,  and  the  highest 
development  schedule  growth  exceeding  its  plan  by  180  percent  (Tyson,  Harmon,  & 
Utech,  1994,  p.  S-2).  The  researchers  also  examined  the  characteristics  of  programs 
with  the  highest  and  lowest  schedule.  Those  often  with  the  highest  schedule  com¬ 
monly  had  high  concurrency  of  interrelated  activities  (e.g.,  overlap  of  development 
and  production).  Additionally,  the  researchers  stated  that  keys  to  preventing  schedule 
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growth  in  development  are  technical  realism  and  a  willingness  to  make  tradeoffs  (Tyson, 
Harmon,  &  Utech,  1994,  p.  S-2). 

Another  finding  from  this  research  was  that  the  major  determinant  of  development 
schedule  growth  is  an  increase  in  quantity  (i.e.,  the  need  to  produce  more  items  for 
testing  than  planned)  (Tyson,  Harmon,  &  Utech,  1994,  pp.  S-5-S-6).  Contrary  to  the 
1990  RAND  study,  the  researchers  found  a  positive  correlation  between  cost  growth 
and  schedule  growth  in  both  development  and  production  (Tyson,  Harmon,  &  Utech, 
1994,  p.  S-6;  Drezner  &  Smith,  1990,  p.  45). 


PREDICTIVE  SCHEDULE  DRIVERS  IN  LITERATURE 

In  Table  1 ,  we  provide  a  cross  tabulation  of  predictive  schedule  drivers  by  study. 
This  tabulation  reveals  that  the  main  drivers  that  prove  beneficial  in  determining 
schedule  variance  are  technical  issues,  the  use  of  competition,  contract  type,  and  the 
existence  of  prototyping.  Although  the  studies  highlighted  differ  in  the  number  of 
programs,  the  source  of  data,  and  methodologies  used,  they  prove  beneficial  in  pro¬ 
viding  insight  into  possible  predictors  a  researcher  could  use  in  estimating  a  program’s 
schedule  duration.  This  in  turn  could  reduce  the  risk  of  underestimating  a  program’s 
completion,  thereby  minimizing  additional  costs  incurred. 


CONCLUSION 

In  this  article,  we  reviewed  a  multitude  of  studies  that  examined  numerous  data¬ 
bases  and  performed  a  variety  of  statistical  procedures,  all  in  the  pursuit  of  explaining 
and  predicting  schedule  duration  and  variance.  It  is  from  these  studies  that  we  can 
identify  the  characteristics  that  drive  acquisition  schedules  and  derive  a  list  of 
predictor  variables.  From  the  past  studies,  we  identify  the  following  potential 
candidates:  program  size,  mission  type,  contract  type,  use  of  competition,  existence 
of  prototyping,  and  number  of  test  articles.  We  also  identify  management  character¬ 
istics,  such  as  the  military  service  and  contractor;  schedule  characteristics,  such  as 
maturity  and  concurrency  measures;  and  other  characteristics  reviewed  in  the  literature, 
such  as  technical  complexity. 

In  an  SOS  approach,  knowing  which  programs  or  systems  may  be  at  risk  of  schedule 
growth  allows  a  PM  to  shift  assets  as  needed  to  ensure,  or  at  least  attempt,  to  keep 
the  overall  system  on  track.  In  addition,  the  potential  schedule  drivers  identified  in 
Table  1  are  just  not  limited  to  thinking  in  terms  of  an  individual  system.  It  is  definitely 
conceivable  that  each  individual  system  may  not  be  so  technologically  complex 
as  to  warrant  thinking  that  it  may  be  a  potential  candidate  for  schedule  slippage  due 
to  complexity  reasons.  However,  when  taken  in  the  aggregate,  the  overall  system  may 
very  well  be  then  prone  to  slippage.  Therefore,  researchers  could  also  look  to  the 
factors  identified  in  the  table  with  respect  to  modeling  schedule  slippage  for  an 
overall  SOS. 
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ENDNOTES 


1.  There  are  five  phases  in  the  life  cycle  of  a  NASA  major  system  acquisition: 

a.  Phase  A  (Preliminary  Anaysis). 

b.  Phase  B  (Definition). 

c.  Phase  C  (Design). 

d.  Phase  D  (Development). 

e.  Phase  E  (Operations). 
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